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ABSTRACT 



Development of Quantitative Models of the Educational Process 

MARTIN STANKARD, JR. 
Management Science Center 
University of Pennsylvania 
Philadelphia, Pennsylvania 



The lack of tested models of the instructional process is the 
prime hindrance to significant management science in educational 
research. The nature of education, with long time lags between 
ultimate performance and present activity, makes the modeling most 
difficult. There are several efforts which have been made to develop 
quantitative models of the educational process. This paper reports 
on only one of them. The model is being developed for future inclusion 
into a school district budgeting and program planning model. The 
history of the model's development as well as some of the problems 
encountered are discussed. Practical implications for further man- 
agement science in educational research are also drawn. 



Introduction: 



There is a missing link in the application of management science 
to problems of educational managers. This missing link is a set of 
tenable models which relate management controllables or decision 
variables to educational outcomes. The existence of the missing link 
causes a gap between what management science should do in educational 
research and what it is being, done. 

I feel that the largest contributions to be made by management 
scientists will come through working with top management of educational 
enterprise. We should be helping them control their systems more 
effectively. This implies that we should be formulating their decision 
problems so that a form of analysis can be applied. If we are to analyze 
the decision and control problems of top managers, much work has to be 
done on developing an understanding of how the controllable portions of 
the educational process relate to measureable objectives. 

Just imagine for a moment that industrial operations were carried 
out without the notion of profit and that least cost operations are not 
always justified. How many of our standard management science 
applications would still hold? Very few, I believe. Perhaps the work in 
information systems would stand but little else. Formulations which would 
still work in this poorly defined situation would be quite minor in terms of 
their importance to a whole enterprise. 

Without filling in the missing link the impact of management science 
on educational systems is going to be limited. In the meantime, managers of 
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educational systems can get some indirect benefit from operations research. 
Application of management science in education can help in drawing better 
school attendence boundaries, in improving forecasts of population and 
enrollment or in building more effective planning systems. I believe, however, 
that these contributions will not satisfy those educators who really need 
much more fundamental help. 

The reasons for the difference between what should be done and 
what is being done are many. Come of them sire common to the practice 
of management science in any organization, and some are, I believe, 
unique to the area of educational management. Problems in carrying out 
management science in education range from lack of support in terms of 
well -qualified personnel and funds to lack of data and a lack of belief that 
the models developed are good models. 

In the remainder of this paper I would like to review the development 
of a particular model of the educational process. With that case as a 
background we have a better basis on which to discuss some of the problems 
which hinder management science in educational research 
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A Case History of an Attempt at Quantitative Modeling of the Education Process 
For several years a group at the University of Pennsylvania headed 
by Roger Sisson has been trying to use operations analysis in educational 
systems. Because of this experience we are enthusiastic about the 
possibilities of operations analysis (management science) in educational 
systems but we do not expect a revolution in the near future. I believe that 
we can identify many problems which are impeding management science 
in educational systems. Some of these problems are discussed later. 

Background and the Starting Point 

Our interest in educational performance models started as a result 
of a simulation study which we did for the School District of Philadelphia. * 

We built and tested an aggregate simulator of the operating and financial 
side of the system. Then we ran experiments on the simulator to estimate 
the sensitivity of district wide costs and other factors to changes in operating 
policy and capital budget parameters. The result of this effort was a series 
of presentations to planning and financial personnel in the system. 

The reaction to the model and the studies which we ran was highly 
favorable at first. But after management became familiar with the model 
they began to ask questions which went completely beyond it. Generally 
the questions were about the quality or educational benefits which would 
result from alternative operating and capital policies. These questions 

Roger L. Sisson: "Some Results of a Simulation of An Urban School 
District, " University of Pennsylvania, Management Science Center Report 
#042467, March, 1967. 
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were the starting point in onr search for the missing link. 

Roger Sisson has discussed modeling of the educational process. ** 

He describes the research and modeling process in terms of three levels: 
the experimental or "no theory" approach, the "thermodynamic" approach 
and the "real theory" or basic approach. The "no theory" approach consists 
of experimentation with prototype systems. "Thermodynamic" modeling 
consists of stating hypothesized relationships between inputs and outputs 
and evaluating parameters by regression. The real theory approach 
consists of building models based upon concepts which apparently explain 
a wide class of phenomena. The successive levels also involve progressively 
higher levels of abstraction. 

We began our attempt to build a quantitative model of the education 
process at the "thermodynamic" level. (The name "thermodynamic" really 
came after we had the model built. ) The educational research literature 
abounds in empirical studies which are the basis for proposing relationships 
of a "thermodynamic" model. Enough has been learned from these studies 
to enable a level of abstraction and modeling above that of plain experimentation. 

The starting point was a definition of education as a communication 
process. Each of the major controllable and environmental variables which 
were hypothesized to be relevant to the process were included in a non- 
linear mathematical model. The final relationship involved changes of 

Roger L. Sisson: "Can "We Model the Education Process?" Address 
before the National Conference on Operations Analysis in Education, U. S. 

Office of Education, Washington, D s C. , November, 1967. To be published 
in Socio-Economic Planning Sciences. 
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composite achievement test scores as a function of teacher time per 
student, books per student, space per student, and level of parental 
education of students. We refer to this as the SD 1. 5 model. 

Tests of the model are hard to obtain because there is little 
variation in the variables from school to school and we were working with 
aggregate data on a school by school basis. In spite of these difficulties 
this first model was generally a better predictor of achievement test scores 
changes than linear regressions on the same data. 

Toward a Theoretical Model. . . . 

The next step in the effort resulted from building a very detailed 
simulator of school district operations from the educational program point 
of view. * The achievement prediction process is to be contained explicitly 
in this simulation. We wanted a much richer model of the educational 
process because of this simulator's detailed representation of educational 
program activity. 

A second feature of this model is that we have tried to base it as 
much as possible on the available theories of learning and intellectual 
development. This model which we call "ACHIEV" represents a step away 
from the "thermodynamic" level of modeling toward the "basic level. " 

It is not yet an entirely theoretical model, however. ACHIEV incorporates 
some elements of theory but also relies on hypothesized functional 

* 

Miguel Szekely, Martin Stankard and Roger Sisson: "Design of a Planning 
Model for an Urban School District. " Socio-Economic Planning Sciences, 
Vol. 1, 1968, Pergamon Press (in preparation). 

A program is the creation of an instructional environment for a group of 
students. 
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relationship among some variables. 

The ACHIEV model is still tentative. We have not yet programmed it 
for the computer. The educational system simulator which it complements 
(SDTwo) is programmed and running. The ultimate goal is to use the com- 
pleted simulator (with performance evaluation included) in program planning 
for large school districts or collections of smaller districts. 



Requirements for the New Model .... 

The present form of the model provides 

a distribution of achievement for each group of students 
whose program mix has been simulated during a simulated 
year. The program mix is the group of educational programs 
which the student actually experienced that year. 

This is based upon: 

. The achievement distribution for that group of students 
at the end of the previous simulated school year. 

* Information about the student group's grade level, socio- 
economic background, home and school areas (in the 
event of voluntary transfers or busing) and other students 
data from the simulator. 

. Information from the simulator about programs given to 
the students and the extent to which each program was 
fully implemented; (pctapp) a variable from the simulator 
which equals the percentage of required funds applied to the 
target group in each program. 

- teachers required vs. teachers available 

- space required vs. space available. 

In the past, the "measure of performance" has been the change of 
score on an achievement type test over a period of time. * We have used this 
mainly by default. There does not appear to be any practical alternative to 



*Martin Stankard and Roger L. Sisson, "On the Modeling of Relationships 
between Performance and Resource Management in an Urban School District, " 
presented at the TIMS-ORSA Joint Meeting, San Francisco, May 1968. 
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this at present. Achievement test scores do not provide any information on 
students' attitudes or on the "affective domain" of performance. Many educators 
whom we have contacted claim that the affective or attitudinal performance con- 
siderations are the more important performance areas. 

The approach to performance prediction which we are proposing here 
does not include the affective domain. We are not specifying a particular per- 
formance instrument as being "the" performance measure. Instead we have a 
general achievement index in mind, something on the order of a grade point 
average. The desire is to gauge how close a particular simulated policy (group 
of programs) comes to allowing each group of students to achieve their best. 

The SDTwo model provides a much richer background for making these 
performance evaluations than did our first Simulator SDOne or SD 1. 5. Con- 
tinuing with the SD 1. 5 approach, we view education as a communication process. 
Now the simulation of individual educational programs for each group of students 
provides more information on which to base a prediction of expected student 
performance. To use the information available we must (1) describe each pro- 
gram in terms of the communication which it is expected to produce, (2) descri 
how the communication is absorbed by the students and (3) relate this finally to 
an achievement or performance index. 
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An Outline of the ACHIEV Model .... 

The best way to describe the ACHIEV Model is to outline the steps neces- 
sary for applying it. A more detailed discussion is included in Appendix I. The 
model is referred to a specified group of students. The steps in modeling the 
process of education for this group are: 

1) The characteristics of this group of students are defined in 
terms of variables relevant to estimating the group's probable inter- 
action with the educational programs applied. This list includes the 
group's age, socio-economic background and attendance behavior, etc. 

2) Define all programs to be applied to the group. Ideally the ob- 
jective of each program is specified first. Then the process of the pro- 
gram is defined. (How much student time is allotted to the program 

and how' much of that allotted’ is Spent in ^ecture^, films, ‘class ‘discussion, 
with tutors, etc?) 

3) Based upon the characteristics of the students and the proposed 
process of each program a breakdown of program activity is estimated. 
This consists of estimating the proportion of program time spent in 
each of several states. 

4) The proportion of program time in each state is weighted and 
then summed across states within a program. The resulting index for 
each program is aggregated across programs. 

5) The index of participation in the educational process is then 
aggregated over time by a smoothing function. The final index sum- 
marizes the program exposure of the group of students. We call this 



index "Richness. " 



6) We assume that the achievement index for a student in the group 
is correlated from year to year. We also assume that two consecutive 
year's achievement index will be jointly distributed with the normal pro- 
bability distribution function. The information about the educational ex- 
perience of the students in this group (the Richness variable) is then used 
to modify the parameters of this distribution. 

The approach to modeling the educational process represented by the 
ACHIEV Model is only an indication of the direction in which quantitative model- 
ing of the educational process can develop. 

The model as it stands now is primarily useful as a generator of hypoth- 
eses and as a guide to the types of data which should be gathered for validating 
models of the educational process. At its present stage of development it is 
hard to class the model as quantitative. Until all of the data required is avail- 
able I suppose that we should refer to this type of model as "semi -quantitative. " 



Problems in Quantitative Modeling of the Education Process 

The major problems which we have encountered in trying to model the 
education process may be grouped into three classes: 

1) problems in the development of theory and model structure 

2) difficulty in validating any models produced and 

3) problems in implementing models to improve system perform- 
ance. 

Problems encountered in structuring models of the educational process 
generally depend on finding a definition of the process which will be acceptable 
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to the ultimate user of a model. It becomes apparent after you try to model 
the education process that the business of education is not very well defined. 
One manager may accept a definition while his colleague will not accept it at 
all. In part this is due to a lack of operationally defined goals and objectives 
and in part it is due to the inherent complexity of the educational process. 

The second area where trouble lurks is in validating models. The 
principal problem here is lack of data. These first two problem areas are 
discussed more fully below. 

Finally, problems in implementing models are encountered. These 
are mainly the product of failures because of the first two trouble spots men- 
tioned. Decision-makers commonly do not believe in the models either be- 
cause they do not conform to some definition of the education process or be- 
cause they are not fully validated. 

1) Development Problems 

a. Lack of Operational Goals and Objectives .... 

Some educational managers have not decided "What 
business they are in. " Education then becomes many things 
to many people. If someone is bold enough to state a set of 
operational criteria for an educational unit, for example, 
a school district, this is the signal for a good deal of criti- 
cism and controversy. 

The problem of setting up performance measures 
gets harder to handle as we look at larger and larger segments 
of an educational system. It is one problem to evaluate the 
performance of the process at the classroom level, and a more 
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complicated problem to do so adequately at the school or 
district level. The quantities which we measure on one student 
may not properly be aggregated across groups. The summari- 
zation as we move from individual to class to school may actually 
average out the information on performance of the process, 
b. The Complexity of the Educational Process .... 

Part of the problem arises from the nature of the 
process of education. First, public education is dealing with 
a special subpopulation, the children. The intellectual capa- 
bilities of children and the ways in which these capabilities 
change are not really understood. Guilford, * for example, 
has gathered evidence which suggests that there may be over 
120 distinguishable types of intellectual capability. When we 
add to this a time dimension we have a very complicated situa- 
tion. Some capabilities may be best influenced at different 
times in an individual’s growth. 

Formal public education for a student usually lasts 
for about twelve years. Except for the continuity of parental 
influence, the individual student is not exposed to a single 
twelve year process but to a sequence of twelve or more one 
year processes. Each of these is under the tactical control of 
a different manager (the teachers). The management of a stu- 
dent’s education as a single twelve year trajectory is not common. 
Actually the educational process is often managed as though 
there were two educational processes, "primary education" and 



* Guilford, J. P. , "Intelligence Has Three Facets;" Science, May 10, 1968. 
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"secondary education", or even as thought there were twelve, 
"first grade education, second grade education, ..." and so on. 
Many real dynamic factors are ignored in this way and while the 
management job is feasible, it is probably not nearly optimal. 

2) Validation Problems 
a. Lack of Data .... 

One of the more frustrating problems in quantitative 
modeling of the educational problems is the lack of good data 
on process variables together with related data on input and 
output. Process data such as text utilization, teaching time 
breakdowns, utilization of various audio-visual devices, amounts 
of student -teacher and student -student interaction, etc. , must 
be used in order to test and revise models of the process. As 
data becomes available I believe that we can continuously im- 
prove our ability to explain output (however it is described) 
given measurements on input. 

It is important to develop sources of data which in- 
clude not just input or process, or output variable, but all 
three. Often school districts maintain elaborate records on 
output (as measured by achievement tests) but fail to record 
the process conditions which presumably account for the 
changes in achievement scores. In other cases, input is mon- 
itored but it cannot be related to output because process data is 
not collected. 

Judging from their data sources educational managers 
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in the past have not appreciated the value of good information 
in decision making. They have not developed the types of data 
collection and reporting systems which are usually found in 
larger industrial firms. It is doubtful that this situation will 
change much in the near future. 

We can draw a useful analogy here between economics 
and education. Before the advent of macro -economics based 
Keynesian theory the data gathered on economic processes were 
nearly hopeless as explicators of economic phenomena. The 
theories of macro -economics gave Simon Kuametgr the basis on 
which to establish the national income accounting system. This 
data base has greatly aided the testing and development of 
models of economic processes at work. 

Perhaps in the educational processes we need a sim- 
ilar rationale for deciding on relevant variables. We need 
some macro -theory of education upon which we could build a 
data base of educational information. Perhaps not on a national 
level, however. 

Insufficient Resources Devoted to Educational Research .... 

The continuing lack of stable and substantial support for all educational 
research is one of the largest barriers to significant management science in 
educational research. The support must be in terms of qualified personnel, and 
adequate and stable financial support for extended periods of time. Research 
appears to be one of the few areas of education where we are likely to find econ- 
omies of scale. It appears that present educational research support and talent 
is so diversified that the realization of scale economies is not very likely. 



A Concluding Remark 
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Recently I have been working with Computer Assisted Instruction, I 
feel that this medium can be a great help in the job of developing the types of 
models of the educational process which we need. The data gathered as a by- 
product of C. A. I. is very useful in modeling, even though it does not cover the 
entire process. 

I believe that C. A, I. can be used much more effectively as a research 
tool if closer attention is paid to the need for research data. I propose a long- 
itudinal application of C. A. I. 

Briefly, this research project would involve installing suitably program- 
med C„ A. I. in grades 1, 2, 3, 5, 7, 9, 11, and 12. This would be done in a sample 
of schools or a whole school district and maintained for a period of at least 
twelve years. The project would be conducted by a consortium consisting of 
research representatives of universities, the school district involved, the 
computer hardware and software suppliers, and the research sponsor. 

The essential feature of this proposal is that detailed data would be 
gathered on a population of student over their entire educational history. This 
would provide a base for research inquiry which is unrivaled. Careful longi- 
tudinal studies could then be conducted. 

A less sophisticated proposal, but possibly just as useful in that school 
district management and research teams should begin to broaden their disciplin- 
ary backgrounds. I advocate a wider use of industrial engineers and other 
trained investigators in studying classroom interactions. There is entirely too 
little understanding of how the process of education actually is carried on at the 
classroom level. Much of the present understanding is anecdotal in nature or 
relies on memories of personal experiences in years gone by. 



Appendix I 
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In this appendix we discuss the structure of the ACHIEV model in more 
detail. Mr. Charles Goldman of the University of Pennsylvania has partici- 
pated in the development of the model to this point. We are still working to 
refine the model and expect to have it programmed for experimentation aud 
ultimately for validation. 

The simulator of which this model will ultimately be a part uses edu- 
cational programs as its basic package of education process. As a result the 
effect of education on the performance of a student group is modeled on a 
program by program basis. In proposing this model we have considered the 
problems of dependence among programs in their effects on achievement and 
the question of the validity of aggregating certain quantities across programs. 
We do not have satisfactory answers to these particular problems at this time. 

The discussion of this model begins with the relevance of programs to 
performance. The representation of achievement as a function of accumulated 
educational process is proposed and finally the bivariate normal model of 
year to year achievement is suggested. 

Programs and Their Relationship to Achievement .... 

A program is the creation of an instructional environment for a group 
of students. Within the environment the students may experience stimuli 
designed to instruct or motivate them. The organized presentation of stimuli 
and the process of providing students with a reservoir of experiences is the 
basis for modeling their expected educational performance. 

Within a particular environment certain responses are permitted or re- 
quired of students. The responses may be spontaneous answers in a classroom 
putting answers on a test paper, being required to answer questions by a 



teacher, etc. The question of the quality of the responses (whether right or 
wrong, good or bad, etc. , is neglected in this note: we are also neglecting 
the qualitative aspect of the stimuli). The only possible justification for this 
omission is that we are proposing a first approach. The main explanatory 
variables upon which we are trying to base performance forecasts are levels 
of stimulus and response encountered by students as a result of a period of 
education. The underlying premise is that the stimuli and responses are 
indicative of the amount of communication in which the students participate. 

The performance of a simulated group of students will depend not only 
upon the communication which goes on in a particular simulated school year, 
but also it will depend upon their past educational experiences. Before be- 
ginning to represent what we have said up to now we will sketch a procedure 
for assessing the relationship between a program and performance. 

Steps to Relate Programs to Performance 

An idealized procedure for relating programs to performance would 
proceed as follows: 

1. Identify and specify the target group of students. This 
consists of stating their socio-economic backgrounds, past achieve- 
ment grade and age. The rest of the procedure is referred strictly 
to this group of students. 

2. Specify the ideal educational process involved in this 
program. Beside resource usage information such as the student 
teacher ratio, use of audio-visual devices, and field trips and special 
lectures the learning activity of the program should be specified. 
Based upon the ideal process the program activity should be broken 
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down two ways: 

a) teacher controlled 

b) student controlled 
Activity oreakdown: 

3. The breakdown of program activity is a judgemental process. It 
would have to be done by an experienced teacher who is familiar with the 
target group of students. This expert teacher would estimate the proportion 
of the time that the target group of students spends in state i under ideal 
process conditions (2 above). 

ACTIVITY BREAKDOWN 

STUDENT 

CONTROLLED 







HIGH response 


LOW response 






(i=l) 


P 2 ( i=2) 




HIGH 

stimulus 


e. g. small group 
tutoring 
computerized 
instruction 


e. g. Film, Good Lecture, 
Good educational TV 


Teacher 

Controlled 


LOW 

stimulus 


P 3 (i=3) 

e. g. testing, quiz, 
Lab experiment 


P 4 (i=4) 

e. g. Recess 
unsupervised class 



4. Estimate a weight or "value" associated with each of the propor- 
tions identified in the preceding step. We propose that the weight be esti- 
mated from answers to the following questions. "Given that the target group 
is in one of the four states, what is the proportion of the students in the target 
group who are inactive ?" Call this proportion i t., i=l, 2, 3, 4. 

5. Determine the proportion of the target group’s time spent in each 
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program, . Here j = 1, 2 , . . . , NPROG (number of programs). As a con- 
vention the superscript j will always indicate that the variable is referred 
to program j. 

6. For the target group of students compute their " & Richness" 

NPROG 4 . . 

& Richness = E ( E p . [ 1 -tt j . ] 
j=l i=l 1 1 

7. The A Richness calculated in this way represents the impact which 
the programs have on the students' experience if all programs are implemented 
according to their ideal specification. Because of limited resources this is 
unlikely. The effect of insufficient staffing or funding in a program will be 

to change the proportions of time which the target group spends in each of 
the activity classes. Let x be the % of staffing requirements actually met for 
a program; program j. 




The graphs show an hypothetical example of how these might change with 
the proportion of staffing requirements met in program j. Another independ- 
ent variable (pctapp)* might be included. We specify the p^ for each program 

as a function of staffing and pctapp for that program 

• • 
p^ (staffing, pctapp); E p^ (*, pctapp) = 1.0 

*pctapp is the percentage: (funds actually spent in the program) * (funds re- 
quired to carry out the program to ideal specifications) 



I 
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So A Richness becomes 



NPROG . 4 

A Richness = E $ (E p, (staffing, pctapp) • ( 1 - it 
3=1 i=l 1 




The Nature of Achievement over time 

In order to represent the dependence of present performance for the 
target group on past performance we absorb ARichness, computed for a target 
group at the end of a simulated year, into a quantity R, total richness. 

At year n let 

R (n) = a • R (n-1) + 0 • A Richness (n) 

where a and 0 are parameters for each tarcet group . 

(They may be time dependent within a target group) 

Let the distribution of achievement for the target group in question be 

2 

normally distributed with parameters ^ , a ^ ^ 

These parameters are computed from the past achievement distribution for 
that group. For the year in question (year n) the corresponding parameters 



relative position in the achievement distribution for year n-1, and year n. 



Let 



= f h [ # of different programs , R (n) ] 

a n-l 



M* n " ^(n-l) = ^o + ®(n)^ * f 2 / Stu. ) • fg (Socio. Econ. ) 



a n, n-1 ^n, n-1 a n a (n-l) 



p h is a measure of the ''coupling" between an individual student's 
* n, n ™ 
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over the Markov Model in as much as it requires that many fewer parameters be 



bability that a laicbmly selected student is observed in state i at year n-1 and 
state j in year n, then 



To describe the Markov process we also need the probability distribution of 
initial states of the system, namely 



Note that the model is non -Markovian when we allow p to change from year to year.* 
The crucial question about the model is whether or not the functions 

f^(* )> • • • * f ^(' ) can be specified. Without really detailed data on educational 
process it will be impossible to establish these relationships with real confidence. 

We propose to specify these functional forms by subjective means where necessary. 



*Wiiliam Feller, Introduction to Probability Theory and its Applications , Vol. II, 
Chap. Ill, John Wiley and Sons, New York 1966. 




We can compute Markov? type transition probabilities, however, if the 
parameter n-1 does not change for n=*l, 2, 3. . . etc. If we let r„ be the pro- 
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P L 5 I i] = Pli,J ]/P|i] 
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